Risk-Sensitive Control of Markov Decision Processes

This paper introduces an algorithm to determine near-optimal control laws for Markov Decision Processes with a risk-sensitive criterion. Both the fully observed and the partially observed settings are considered, for nite and innnite horizon formulations. Dynamic programming equations are introduced which characterize the value function for the partially observed, innnite horizon , discounted costs formulation. An alternative risk-sensitive formulation is examined, for which there exists a stationary innnite horizon optimal policy. Policy and value iteration algorithms are used to determine such a policy. Finally , the alternative formulation is extended in a natural way to the partially observed setting.